In augmented reality (AR) environments, a user may view an integration of artificial or virtual graphics with the user's natural surroundings. In some early implementations of AR, a user may see graphics displayed arbitrarily amongst or within the user's natural surroundings via, for example, augmented reality goggles. For instance, a graphic of a random butterfly may fly along the view of the AR goggles while the user continues to view his or her natural surroundings, regardless of whether the butterfly has any relevance to anything the user is seeing naturally. In more sophisticated implementations of AR, a user may be able to apply AR features or graphics directly to objects or structures of the user's natural surroundings. For example, an object sitting on a table may be identified and rendered with a different color or different physical attributes in the AR environment. However, augmenting the reality of an object during runtime requires that the object in the environment is identified and re-rendered with the desired characteristics.
Limiting AR to known and identified objects reduces the applicability of AR applications. Techniques, described herein, solve this and other problems.